Implementing selective cache injection

ABSTRACT

A method, system and memory controller for implementing memory hierarchy placement decisions in a memory system including direct routing of arriving data into a main memory system and selective injection of the data or computed results into a processor cache in a computer system. A memory controller, or a processing element in a memory system, selectively drives placement of data into other levels of the memory hierarchy. The decision to inject into the hierarchy can be triggered by the arrival of data from an input output (IO) device, from computation, or from a directive of an in-memory processing element.

FIELD OF THE INVENTION

The present invention relates generally to the data processing field,and more particularly, relates to a method, system and memory controllerfor implementing memory hierarchy placement decisions in a memorysubsystem including direct routing of arriving data into a main memorysystem and selective injection of the data or computed results into aprocessor cache in a computer system.

DESCRIPTION OF THE RELATED ART

Modern computer systems typically are configured with a large amount ofmemory in order to provide data and instructions to one or moreprocessors in the computer systems.

Historically, processor speeds have increased more rapidly than memoryaccess times to large portions of memory, in particular, DRAM memory(Dynamic Random Access Memory). Memory hierarchies have been constructedto reduce the performance mismatches between processors and memory. Forexample, most modern processors are constructed having an L1 (level 1)cache, constructed of SRAM (Static Random Access Memory) on a processorsemiconductor chip. L1 cache is very fast, providing reads and writes inonly one, or several cycles of the processor. However, L1 caches, whilevery fast, are also quite small, perhaps 64 KB (Kilobytes) to 256 KB. AnL2 (Level 2) cache is often also implemented on the processor chip. L2cache is typically also constructed using SRAM storage, although someprocessors utilize DRAM storage. The L2 cache is typically several timeslarger in number of bytes than the L1 cache, but is slower to read orwrite. Some modern processor chips also contain higher level cache, suchas an L3 (Level 3) cache. L3 cache is capable of holding several timesmore data than the L2 cache. L3 cache is sometimes constructed with DRAMstorage. L3 cache in some computer systems is implemented on a separatechip or chips from the processor, and is coupled to the processor withwiring on a printed wiring board (PWB) or a multi-chip module (MCM).Main memory of the computer system is typically large, often many GB(gigabytes) and is typically implemented in DRAM.

Main memory is typically coupled to a processor with a memorycontroller, which may be integrated on the same device as the processoror located separate from the processor, often on the same MCM(multi-chip module) or PWB. The memory controller receives load or readcommands and store or write commands from the processor and servicesthose commands, reading data from main memory or writing data to mainmemory. Typically, the memory controller has one or more queues, forexample, read queues and write queues. The read queues and write queuesbuffer information including one or more of commands, controls,addresses and data; thereby enabling the processor to have multiplerequests—including read and/or write requests, in process at a giventime.

Extensive research and development efforts are invested by the industry,on an ongoing basis, to create improved, innovative solutions formaximizing overall system performance and density by improving thememory system or memory subsystem design and structure.

New data may enter the system through several means including eitherarrival from some external entity, or modification of existing datavalues. Today, with few exceptions, data arriving at a processor isinserted into two potential locations including 1) main memory, insystems with no cache injection support; and 2) processor cache, ifcache injection is supported. If data is placed in main memory inconventional arrangements, then the data must travel over the memory bustwice before it can be processed. If placed in a cache, the data can beevicted before the processor accesses this data in the cache.

A problem of state of the art computer systems is that the processormight not actually need the data (which has come in off the network, orotherwise from IO), or the processor might not need it right now. Datathus inserted into the caches frequently pollutes the caches, since thedata is not immediately needed for computation, and unnecessarily takesup space in the cache.

In a memory system, it is desirable to place data directly into thememory instead of a processor cache. Also it can difficult to know whichprocessor cache data should be placed in non-shared caches if the workhas not been scheduled. Large data block moves that are cache injectedoften would just roll caches and would cause a loss of performance.Small data updates would be very random and most likely occur on datathat is currently in memory and not in a cache.

A need exists for an effective mechanism having the ability to insertarriving data directly into the main memory system and selectivelyinject the data or computed results into a processor cache in a computersystem.

As used in the following description and claims, the terms should bebroadly understood as set forth in the following:

Cache injection is a process of inputting data from main memory into aprocessor cache.

Memory-routed data is data sent directly to main memory withoutinsertion into a processor cache.

SUMMARY OF THE INVENTION

Principal aspects of the present invention are to provide a method,system and memory controller for implementing memory hierarchy placementdecisions and selective injection of data or computed results into aprocessor cache in a main memory subsystem in a computer system. Otherimportant aspects of the present invention are to provide such method,system and memory controller substantially without negative effects andthat overcome many of the disadvantages of prior art arrangements.

In brief, a method, system and memory controller for implementing memoryhierarchy placement decisions in a memory system including directrouting of arriving data into a main memory system and selectiveinjection of the data or computed results into a processor cache in acomputer system. A memory controller, or processing elements in a memorysubsystem, selectively drives placement of data into a level of thememory hierarchy. The decision to place data into the memory hierarchycan be triggered by the arrival of data from an input output (IO)device, from computation, or from a directive of an in-memory processingelement.

In accordance with features of the invention, the data for which aplacement decision is made includes the data which was received,computed, or otherwise changed; and other data or computed results.

In accordance with features of the invention, the memory controller, orin-memory processing elements inject data into the processor cache,after that data, or other data has been held, analyzed, computed, orotherwise changed in the memory.

In accordance with features of the invention, the memory controller, orin-memory processing elements injecting data into the processor cache,further includes updating state information for the injected data tokeep the processor cache from evicting the injected data until theprocessor has accessed or deallocated the injected data. The injectedcache data optionally includes directory bits that are set to preventeviction, and that are cleared when the injected cache data is accessedor deallocated by the processor.

In accordance with features of the invention, the processor optionallysends a trigger or special load or store instruction to promptcomputation on the data in memory.

In accordance with features of the invention, the injection of data orcomputed results from main memory into the processor cache optionallyincludes a processor wake-up or notification.

In accordance with features of the invention, derived data optionally iscreated and pre-loaded into the processor cache, for example, forpotential additional computation.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention together with the above and other objects andadvantages may best be understood from the following detaileddescription of the preferred embodiments of the invention illustrated inthe drawings, wherein:

FIG. 1 is a block diagram of an example computer system embodying thepresent invention;

FIG. 2 is a block diagram illustrating an example memory system forimplementing memory hierarchy placement decisions including directrouting of arriving data into a main memory system and selectiveinjection of the data or computed results into the processor cache inthe computer system of FIG. 1 in accordance with the preferredembodiment;

FIG. 3 is a flow chart illustrating example operations for implementingmemory hierarchy placement decisions including direct routing ofarriving data into a main memory system and selective injection of thedata or computed results into the processor cache in the memory systemof FIG. 2 in accordance with the preferred embodiments;

FIG. 4 is a block diagram illustrating a computer program product inaccordance with the preferred embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following detailed description of embodiments of the invention,reference is made to the accompanying drawings, which illustrate exampleembodiments by which the invention may be practiced. It is to beunderstood that other embodiments may be utilized and structural changesmay be made without departing from the scope of the invention.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

In accordance with features of the invention, a method, system andmemory controller are provided for implementing memory hierarchyplacement decisions including direct routing of arriving data into amain memory system and selective injection of the data or computedresults into the processor cache.

Having reference now to the drawings, in FIG. 1, there is shown acomputer system embodying the present invention generally designated bythe reference character 100 for implementing memory hierarchy placementdecisions in accordance with the preferred embodiment. Computer system100 includes one or more processors 102 or general-purpose programmablecentral processing units (CPUs) 102, #1-N. As shown, computer system 100includes multiple processors 102 typical of a relatively large system;however, system 100 can include a single CPU 102. Computer system 100includes a cache memory 104 connected to each processor 102.

Computer system 100 includes a memory system 106 including a memorycontroller 108 and a main memory 110 connected by a bus 112. Bus 112 isone or more busses that send address/command information to main memory110 and send and receive data from the memory 110. Main memory 110 is arandom-access semiconductor memory for storing data, including programs.Main memory 110 is comprised of, for example, a dynamic random accessmemory (DRAM), a synchronous direct random access memory (SDRAM), acurrent double data rate (DDRx) SDRAM, non-volatile memory, opticalstorage, and other storage devices.

I/O bus interface 114, and buses 116, 118 provide communication pathsamong the various system components. Bus 116 is a processor/memory bus,often referred to as front-side bus, providing a data communication pathfor transferring data among CPUs 102 and caches 104, memory controller108 and I/O bus interface unit 114. I/O bus interface 114 is furthercoupled to system I/O bus 118 for transferring data to and from variousI/O units.

As shown, computer system 100 includes a storage interface 120 coupledto storage devices, such as, a direct access storage device (DASD) 122,and a CD-ROM 124. Computer system 100 includes a terminal interface 126coupled to a plurality of terminals 128, #1-M, a network interface 130coupled to a network 132, such as the Internet, local area or othernetworks, and a I/O device interface 134 coupled to I/O devices, such asa first printer/fax 136A, and a second printer 136B.

I/O bus interface 114 communicates with multiple I/O interface units120, 126, 130, 134, which are also known as I/O processors (IOPs) or I/Oadapters (IOAs), through system I/O bus 116. System I/O bus 116 is, forexample, an industry standard PCI bus, or other appropriate bustechnology.

Computer system 100 is shown in simplified form sufficient forunderstanding the present invention. The illustrated computer system 100is not intended to imply architectural or functional limitations.Although main memory 102 and cache 106 are represented conceptually inFIG. 1 as single entities, it will be understood that in fact these aremore complex, and that cache may exist at multiple different levels, asis known in the art. In particular, main memory subsystem 102 comprisesmultiple modules and communications components. The present inventioncan be used with various hardware implementations and systems andvarious other internal hardware devices, for example, multiple mainprocessors.

In accordance with embodiments of the invention, an example memorysystem 200 is further illustrated and described with respect to FIG. 2.

Referring now to FIG. 2, there is shown an example memory systemgenerally designated by the reference character 200 for implementingmemory hierarchy placement decisions including direct routing ofarriving data into a main memory system and selective injection of thedata or computed results into the processor cache 104 in the computersystem 100 in accordance with the preferred embodiment. In the memorysystem 200, as indicated in a block 204 data is received from an 10device or data is exchanged over the network, such as network 132 inFIG. 1.

Memory system 200 includes a memory controller 208, which makes memoryhierarchy placement decisions, and a main memory 210 arranged forin-memory processing. Memory system 200 includes in-memory processingelements 212 that are contained in the memory controller 208, the mainmemory 210 or in a separate chip, for example, attached to the memorycontroller 208 or the main memory 210. The in-memory processing elements212 provide capability for performing computation or other analyses onmemory-routed data, and determining what to do with data which arrivesat the system 200.

Referring also to FIG. 3, there are shown example operations forimplementing memory hierarchy placement decisions including directrouting of arriving data into a main memory 210 and selective injectionof the data or computed results into the processor cache 104 in thememory system 200 in accordance with the preferred embodiments. Asindicated in a block 300, operations start and data generated from thein-memory processing element, data received from an 10 device or dataexchanged over the network as indicated in a block 302.

As indicated in a block 304, a decision is made to route data to cacheor directly to main memory. The decision at block 304 to place the datainto the processor cache optionally is based upon the data being neededimmediately in a next step. Alternatively the decision at block 304 toplace the data directly into the main memory optionally is based uponthe data being archival, and may never need to go into the cache. Thedecision at block 304 to place the data directly into the main memoryoptionally is based upon the data coming in a packed form and needing tobe distributed to different data structures before computation, the datacan be stored in the active memory, and redistributed into the memory ofthe data structure through a program stored in the memory itself. Thedecision at block 304 to place the data directly into the main memoryoptionally is based upon the data not being needed immediately in a nextstep, and further processing being needed. For example, at block 304 ifthe data needs to be merged with data from other parts (or this part) ofthe network before use, the active memory can perform the merge andplace the data into the processor cache. Also, at block 304 for example,if the data needs to be merged with data from other parts (or this part)of the network before use, the active memory can be triggered to performthe merge on demand by the processor and return the computed values.

As indicated in a decision block 306, checking for memory routed datasent directly to the main memory, without insertion into cache, isperformed. When the data is not memory routed data, then the data isrouted to cache, and held until the data is accessed or deallocated asindicated in a block 308. The injected cache data includes, for example,directory bits that are set to prevent eviction, and that are clearedwhen the data is accessed or deallocated by the processor.Alternatively, the injected cache data includes set state information toprevent premature eviction, such as a most-recently-used (MRU) state togive the injected cache data the longest life span in cache.Alternatively, the injected cache data is provided in cache injectionrange registers to prevent premature eviction to keep the injected cachedata in cache.

When the data is memory routed data, then checking for action on thedata is performed as indicated in a decision block 310. When a datare-route to cache is identified, then the data is rerouted to cache andheld until the data is accessed or deallocated as indicated in a block312. When no action is identified at decision block 310, then theoperations end as indicated in a block 314. When processing of the datais identified at decision block 310, then computation or other analysison the data is performed as indicated in a block 316.

As indicated in a block 318, a predefined operation, such as, that mayinclude computations and other decisions may be performed independentlyin the main memory system, the injection process may include a processorwake-up or notification function, such as based on monitoring of somelocation which indicates data is ready. At block 318 the injected cachedata is held until the data is accessed or deallocated. Injection ofdata or computed results from main memory into the processor cache atblock 318 can be speculative, such as prefetch, since the process mightnot actually use the value the memory entities put into the processorcache. Also, at block 318 for example, derived data can be created,which is then pre-loaded into the cache for potential additionalcomputation. At block 318 for example, injection of data or computedresults from main memory into the processor cache might be in responseto the processor sending a trigger, such as a special load or storeinstruction, to prompt computation on the data in memory and depositfiltered results back into the processor cache for potential additionalcomputation.

Referring now to FIG. 4, an article of manufacture or a computer programproduct 400 of the invention is illustrated. The computer programproduct 400 is tangibly embodied on a non-transitory computer readablestorage medium that includes a recording medium 402, such as, a floppydisk, a high capacity read only memory in the form of an optically readcompact disk or CD-ROM, a tape, or another similar computer programproduct. Recording medium 402 stores program means 404, 406, 408, and410 on the medium 402 for carrying out the methods for implementingmemory hierarchy placement decisions of the preferred embodiment in thesystem 100 of FIG. 1, and memory system 200 of FIG. 2.

A sequence of program instructions or a logical assembly of one or moreinterrelated modules defined by the recorded program means 404, 406,408, and 410, direct the computer system 100, and memory system 200 forimplementing memory hierarchy placement decisions of the preferredembodiment.

While the present invention has been described with reference to thedetails of the embodiments of the invention shown in the drawing, thesedetails are not intended to limit the scope of the invention as claimedin the appended claims.

What is claimed is:
 1. A method for implementing memory hierarchyplacement decisions in a memory system in a computer system comprising:routing arriving data directly into a memory system; selectivelyinjecting the data into a processor cache; and using one of a memorycontroller and a processing element in the memory system, selectivelydriving placement of the data into a level of the memory hierarchy. 2.The method as recited in claim 1 wherein selectively injecting the datainto the processor cache includes holding the injected data untilaccessed or deallocated by a processor.
 3. The method as recited inclaim 2 includes providing a directory bit with the injected data toprevent eviction, and clearing the directory bit with the injected databeing accessed or deallocated by the processor.
 4. The method as recitedin claim 1 includes one of triggering a decision for selectivelyinjecting the data or computed results into the processor cache basedupon arrival of data from an IO device; triggering a decision forselectively injecting the data or computed results into the processorcache based upon a result of computation using the data; or triggering adecision for selectively injecting the data or computed results into theprocessor cache based upon a directive of an in-memory processingelement.
 5. The method as recited in claim 1 wherein selectivelyinjecting the data into the processor cache includes updating stateinformation for the injected data into the processor cache.
 6. Themethod as recited in claim 1 wherein selectively injecting the data intothe processor cache includes sending a processor notification forprocessing injected data.
 7. The method as recited in claim 1 whereinthe data includes one of changed data, derived data, computed results,and generated data, and includes selectively injecting one of thearriving data, changed data, derived data, computed results andgenerated data into the processor cache.
 8. The method as recited inclaim 1 wherein selectively injecting the data into the processor cacheincludes sending processor prefetch information into the processorcache.
 9. The method as recited in claim 1 includes rerouting computedor changed data into the processor cache.
 10. A memory systemcomprising: a memory controller for implementing memory hierarchyplacement decisions, a processor; a processor cache coupled to saidmemory controller and said processor; a main memory coupled to saidmemory controller and said processor cache; an in-memory processingelement coupled to said main memory and said memory controller; saidmemory controller selectively routing arriving data directly into saidmain memory; and said memory controller selectively injecting the datainto said processor cache; one of said memory controller and saidin-memory processing element, selectively driving placement of the datainto a level of the memory hierarchy.
 11. The system as recited in claim10 includes said processor sending an instruction to one of said memorycontroller and said in-memory processing element, for processing thedata.
 12. The system as recited in claim 10 wherein said memorycontroller selectively injecting the data into the processor cacheincludes said memory controller setting directory bits for holding theinjected data until accessed or deallocated by a processor.
 13. Thesystem as recited in claim 10 wherein said memory controller selectivelyinjecting the data into the processor cache includes said memorycontroller providing state information for the injected data into theprocessor cache for holding the injected data until accessed ordeallocated by a processor.
 14. The system as recited in claim 10includes said memory controller receiving a decision trigger forselectively injecting the data or computed results into the processorcache based upon arrival of data from an IO device; receiving a decisiontrigger for selectively injecting the data or computed results into theprocessor cache based upon a result of computation using the data; orreceiving a decision trigger for selectively injecting the data orcomputed results into the processor cache based upon a directive of saidin-memory processing element.
 15. The system as recited in claim 10includes said in-memory processing element performing computation andanalysis for processing arriving data and data generated from saidin-memory processing element.
 16. The system as recited in claim 10wherein said memory controller selectively injecting the data into theprocessor cache includes said memory controller sending a processornotification for processing injected data.
 17. The system as recited inclaim 10 wherein said memory controller selectively injecting the datainto the processor cache includes said memory controller reroutingderived data into the processor cache.
 18. The system as recited inclaim 10 wherein said memory controller selectively injecting the datainto the processor cache includes said memory controller sendingprocessor prefetch information into the processor cache.
 19. The systemas recited in claim 10 wherein the data includes one of received data,changed data, derived data, computed results, and generated data, andincludes said memory controller selectively injecting one of thereceived data, changed data, derived data, computed results andgenerated data into the processor cache.
 20. The system as recited inclaim 19 includes said in-memory processing element selectivelyinjecting one of the received data, changed data, derived data, computedresults and generated data into the processor cache.